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Abstract 

This  paper  concerns  scheduling  policies  in  a  surveillance  system  aimed  at  detecting 
a  terrorist  attack  in  time.  Terrorist  suspects  arriving  at  a  public  area  are  subject 
to  continuous  monitoring,  while  a  surveillance  team  takes  their  biometric  signatures 
and  compare  them  with  records  stored  in  a  terrorist  database.  Because  the  surveil¬ 
lance  team  can  screen  only  one  terrorist  suspect  at  a  time,  the  team  faces  a  dynamic 
scheduling  problem  among  the  suspects.  We  build  a  M/G/l  queue  with  two  types 
of  customers — red  and  white — to  study  this  problem.  Both  types  of  customers  are 
impatient,  but  the  reneging  time  distributions  are  different.  The  server  only  receives 
reward  by  serving  a  red  customer,  and  can  use  the  time  a  customer  has  spent  in  the 
queue  to  deduce  its  likely  type.  In  a  few  special  cases,  a  simple  service  rule — such  as 
the  first-come-first-serve  rule — is  optimal.  We  explain  why  the  problem  is  in  general 
difficult,  and  develop  a  heuristic  policy  motivated  by  the  fact  that  terrorist  attacks  are 
rare  events. 

Keywords:  homeland  security,  counterterrorism,  multiclass  queue,  reneging,  dynamic  schedul¬ 
ing. 
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1  Introduction 


Terrorist  attacks — such  as  bombing,  assassination  of  political  figures,  and  release  of  poison 
gas  in  a  crowd — are  serious  threats  in  many  regions  of  the  world.  A  significant  terrorist 
attack  occured  in  1972  at  a  ticket  counter  in  Lod  International  airport  near  Tel  Aviv,  Israel, 
where  a  three-man  hit  squad  from  the  Japanese  Red  Army  killed  26  people  and  injured  78 
more.  More  recent  examples  include  the  9/11  attacks  in  2001,  the  Bali  bombings  in  2002 
and  2005,  and  the  London  bombings  in  2005.  Numerous  instances  in  the  past  suggest  that 
terrorists  often  aim  their  attacks  at  crowded  locations — such  as  restaurants,  transportation 
terminals,  popular  tourist  spots,  political  rallies,  and  subway  stations — to  create  chaos  and 
cause  damage.  The  consequences  of  such  terrorist  attacks  are  casualties,  damaged  property, 
and  a  major  disruption  of  daily  life. 

Response  actions  for  mitigating  and  countering  terrorist  attacks  include  political  and 
social  policies  that  aim  at  deterring  recruitment  of  terrorists,  as  well  as  protection  of  potential 
targets  by  police  or  military  forces.  While  the  authorities  spend  considerable  resources  going 
after  the  sources  of  such  attacks — the  terrorist  organizations  and  their  infrastructure — it  is 
still  important  to  have  the  ability  to  thwart  terrorist  attacks  by  timely  detection  and  effective 
response.  In  this  paper,  we  focus  on  that  last  line  of  defense — the  problem  of  detecting,  as 
early  as  possible,  a  developing  terrorist  attack  on  a  public  target. 

We  consider  a  large  public  area — henceforth  called  arena — where  people  can  come  and 
go  freely,  such  as  an  airport  lobby  or  a  popular  tourist  attraction.  An  array  of  video  cameras 
monitors  the  arena  and  feeds  real-time  video  streams  to  a  control  center,  where  a  security 
team  screens  people  in  two  phases.  In  the  first  phase,  people  entering  the  arena  are  examined 
visually  and  each  person  is  immediately  put  into  one  of  two  groups:  nonsuspects  and  sus¬ 
pects.  Only  suspects  are  subject  to  the  second-phase  screening,  which  includes  taking  their 
biometric  signatures  (such  as  face  structure,  hair  color,  etc.)  and  running  them  through 
a  terrorist  database  for  comparison.  In  case  of  a  positive  match,  the  suspect  is  classified 
as  a  potential  terrorist  and  security  forces  are  notified  to  take  proper  actions;  otherwise, 
the  suspect  is  reclassified  as  a  nonsuspect  and  the  security  team  moves  on  to  conduct  the 
second-phase  screening  on  another  suspect.  Because  in  the  second  phase,  the  security  team 
can  screen  only  one  suspect  at  a  time,  the  team  faces  a  dynamic  scheduling  problem  among 
the  suspects  with  the  goal  to  maximize  the  probability  of  detecting  a  terrorist  attack  in  time. 
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Two  observations  motivate  our  research  problem.  First,  because  the  second-phase  screen¬ 
ing  takes  time,  the  security  team  may  not  be  able  to  inspect  all  suspects  before  they  leave 
the  arena.  Second,  because  a  terrorist’s  intention  and  action  are  different  from  those  of  other 
people  in  the  arena,  the  distribution  of  the  time  he  spends  in  the  arena  may  be  different  too. 
Consequently,  by  carefully  choosing  which  suspect  to  inspect  next,  one  could  potentially 
increase  the  probability  of  detecting  a  terrorist  attack  in  time.  In  this  paper,  we  develop  a 
queueing  model  with  impatient  customers  of  unknown  identity  to  analyze  this  problem,  and 
draw  insights  into  the  effect  of  scheduling  policies  on  such  a  surveillance  system.  Several 
attempts  have  recently  been  made  to  model  and  analyze  detection  and  response  actions  as¬ 
sociated  with  counterterrorism  and  homeland  security;  for  example,  see  [7,  9,  10,  11,  13,  19]. 
However,  we  are  not  aware  of  any  work  that  addresses  a  closely  related  problem. 

The  contribution  of  this  paper  is  twofold.  From  a  theoretical  standpoint,  we  build  a 
queueing  model  with  impatient  customers  that  describes  the  antiterrorist  surveillance  system. 
There  are  two  types  of  customers — terrorists  and  nonterrorists.  The  novelty  of  this  queueing 
model  is  that  only  one  type  of  customer  (terrorists)  is  worth  serving,  but  the  server  does 
not  know  a  customer’s  identify  until  service  completion.  From  an  application  standpoint 
point,  we  develop  dynamic  scheduling  policies  for  an  antiterrorist  surveillance  system  that 
can  improve  the  probability  of  detecting  a  terrorist  in  a  crowded  area. 

The  rest  of  this  paper  is  organized  as  follows.  In  Section  2,  we  discuss  the  operational 
setting  and  develop  a  queueing  model.  In  Section  3,  we  identify  a  few  special  cases  where 
the  optimal  policy  can  be  explicitly  determined.  In  Section  4,  we  discuss  why  the  optimal 
policy  is  difficult  to  derive  in  general,  and  develop  a  heuristic  policy.  Conclusions  and  future 
research  directions  are  discussed  in  Section  5. 


2  The  Model 

In  this  section,  we  build  a  mathematical  model  to  study  the  second-phase  screening  discussed 
in  Section  1.  Suppose  the  suspects  that  are  subject  to  the  second-phase  screening  arrive 
according  to  a  Poisson  process  with  rate  A.  Each  suspect  is  independently  a  red  customer 
(terrorist)  with  probability  p,  or  a  white  customer  (nonterrorist)  with  probability  1  —p.  It  is 
helpful  to  keep  in  mind  that  p  is  very  small  because  terrorist  attacks  are  rare  events.  Both 
types  of  customers  are  impatient,  and  will  leave  the  arena  after  a  random  amount  of  time 
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regardless  whether  the  second-phase  screening  has  started.  The  time  a  red  customer  spends 
in  the  arena — called  reneging  time  in  queueing  theory — represents  the  time  it  takes  for  a 
terrorist  to  initiate  an  attack.  The  time  a  white  customer  spends  in  the  arena  represents 
the  time  an  innocent  civilian  wonders  in  the  arena.  We  assume  that  the  reneging  time 
distribution  of  red  customers  Fr(-)  can  be  estimated  from  intelligence  and  past  events,  while 
that  of  white  customers  F\y(j  can  be  estimated  from  data. 

The  second-phase  screening  comprises  a  continuous  monitoring  of  the  suspect,  while 
running  the  suspect’s  biometric  signature  through  a  terrorist  database  for  comparison.  In 
our  queueing  model,  the  security  team  is  the  server  who  provides  service — second-phase 
screening — to  customers  one  at  a  time.  The  service  time  follows  a  distribution  function 
Fs(-),  independent  of  the  customer’s  identity.  The  objective  of  the  server  is  to  detect  a  red 
customer  in  time  so  that  the  security  forces  can  take  proper  actions  to  prevent  or  mitigate 
the  attack.  In  other  words,  only  red  customers  are  valuable  for  service.  The  server,  however, 
cannot  tell  the  identity  of  a  customer  until  after  the  service. 

To  define  an  objective  function  of  the  problem,  note  that  whereas  a  customer  waiting  in 
queue  may  depart  the  system  due  to  his  impatience,  a  customer  in  service  will  depart  the 
system  either  due  to  his  impatience  or  due  to  service  completion,  whichever  occurs  first.  If 
the  departing  customer  is  white,  the  process  continues;  in  case  the  departing  white  customer 
was  in  service,  the  server  becomes  available  and  immediately  chooses  another  customer  in 
the  queue  to  serve.  The  process  ends  as  soon  as  a  red  customer  departs  the  system  for  the 
first  time,  which  includes  three  possible  scenarios.  First,  if  the  red  customer  departs  before 
service  is  initiated,  the  server  fails  and  receives  a  reward  of  0,  because  a  terrorist  attack 
takes  place  without  prior  warning  and  the  damage  will  be  at  its  greatest.  Second,  if  the 
red  customer  departs  due  to  service  completion,  the  server  succeeds  and  receives  a  reward 
of  1,  because  the  screening  team  identifies  the  terrorist  in  time  to  prevent  the  attack.  Last, 
if  the  red  customer  departs  while  in  service — due  to  the  initiation  of  an  attack — the  server 
succeeds  partially  and  receives  a  reward  of  r  G  [0, 1],  because  the  security  team  identifies  the 
terrorist  and  can  respond  to  it  quickly.  The  rationale  that  the  surveillance  process  stops  as 
soon  as  a  red  customer  departs  is  that  in  all  three  scenarios  the  police  or  military  force  will 
take  in  charge  immediately — locking  down  the  arena,  evacuating  the  civilians,  etc. — which 
makes  continual  surveillance  irrelevant.  Therefore,  the  objective  of  the  server  is  to  schedule 
the  service  sequence  in  real  time  in  order  to  maximize  the  expected  value  of  the  reward  when 
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the  surveillance  process  ends.  Note  that  the  server  may  still  fail  even  when  the  first  arriving 
red  customer  enters  service,  because  another  red  customer  may  arrive  and  renege  before  the 
first  arriving  customer  departs. 

The  server’s  problem  is  to  decide  which  customer  in  the  queue  to  serve  each  time  the 
server  becomes  available.  Specifically,  we  can  delineate  the  state  of  the  queue  by 

(N)  ^2j  ■  ■  ■  i  tn)i  H  ^  tn , 

with  the  interpretation  that  there  are  n  customers  in  the  queue,  and  the  ?’th  customer  has 
spent  tj  time  units  in  the  queue.  Note  that  we  do  not  need  to  include  the  time  since  the  last 
customer  arrival  in  the  state  space  because  the  customer  arrival  process  is  a  Poisson  process. 
A  feasible  policy  is  a  function  that  maps  a  vector  (H, . . . ,  tn)  to  an  index  i  e  {1, . . . ,  n},  for 
n=  1,2,.... 

In  queueing  theory,  there  is  extensive  research  that  concerns  dynamic  scheduling  of  mul¬ 
ticlass  queueing  networks.  In  a  service  center,  different  classes  of  customers  bring  in  different 
revenue  and  require  different  service  times;  for  example,  see  Miller  [14]  and  Harrison  [5].  In  a 
production  system,  switching  from  one  customer  class  to  the  other  may  require  setup  times; 
for  example,  see  Reiman  and  Wein  [17]  and  Olsen  [15].  For  real-time  scheduling  problems 
involving  impatient  customers,  see  Gaver  et  al.  [2],  Glazebrook  et  al.  [3],  Jouini  et  al.  [8], 
and  the  references  therein.  More  recently,  there  is  a  growing  interest  in  multiclass  queues 
in  heavy  traffic;  for  example,  see  Bertsimas  and  Mourtizinou  [1],  Plambeck  et  al.  [16],  and 
Harrison  and  Zeevi  [6].  The  major  distinction  between  our  model  and  these  earlier  works 
is  that  in  our  model,  a  customer  does  not  reveal  his  identity  upon  arrival,  and  the  server 
can  gather  information  about  a  customer’s  identity  by  studying  how  long  the  customer  has 
spent  in  the  queue.  To  the  best  of  our  knowledge,  our  work  is  the  first  to  address  this  type 
of  problem. 

3  Exponential  Reneging  Time  Distribution 

This  section  presents  the  case  when  both  Fr{  and  Fw  are  exponential.  In  Subsection  3.1  we 
study  the  first- come- first- serve  rule,  and  in  Subsection  3.2  we  study  the  last- come- first- serve 
rule.  In  Subsection  3.3  we  consider  the  random-selection  rule,  and  compare  all  three  rules 
numerically.  Although  our  primary  interest  is  to  study  a  nonpreemptive  service  system, 
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in  Subsection  3.4  we  discuss  a  preemptive  service  system  that  complements  our  theoretical 
results. 

3.1  First-Come-First-Serve  (FCFS)  Rule 

With  the  FCFS  rule,  the  server  always  serves  the  customer  who  has  spent  the  longest  in  the 
queue.  If  the  reneging  time  distributions  for  both  red  and  white  customers  are  exponential, 
the  next  theorem  presents  a  sufficient  condition  for  the  FCFS  rule  to  be  optimal.  Note  that 
the  theorem  does  not  require  the  service  time  distribution  Fg  to  be  exponential,  neither  does 
it  require  the  arrival  process  to  be  a  Poisson  process. 

Theorem  3.1  If  both  Fr  mid  Fw  are  exponential  with  respective  rates  6r  <  Ow,  then  the 
FCFS  rule  is  optimal  for  any  r  G  [0, 1]  ,  for  an  arbitrary  distribution  function  Fs,  and  for  an 
arbitrary  arrival  process. 

Proof:  Consider  an  arbitrary  state  (U, 12, . . . ,  tn)  such  that  ti  >  t2  >  ■  ■  ■  >  tn.  We  first 
want  to  show  that  for  any  policy  that  does  not  start  with  customer  1,  we  can  ford  a  better 
policy  by  starting  with  customer  1.  The  proof  relies  on  an  argument  that  involves  stochastic 
coupling  between  two  sample  paths.  A  reference  to  the  stochastic  coupling  technique  can  be 
found  in  Section  9.2  in  Ross  [18]. 

Let  p{t)  denote  the  probability  that  a  customer  in  the  queue  is  red  if  he  has  spent  t  time 
units  in  the  queue.  Using  Bayes’  rule,  we  can  calculate  that 

'  pFH(t)  +  (l-p)Fw(t)’  U 

where  PrW  =  1  —  Fft(t)  is  the  tail  distribution  function  of  a  red  customer’s  reneging  time, 
and  Fw(t )  =  1  —  Fw(t)  is  that  of  a  white  customer’s  reneging  time.  Because  both  Fr  and 
Fw  are  exponential  with  respective  rates  0R  <  0W,  it  follows  that  p(t)  increases  in  t  (the  first 
derivative  of  Equation  (1)  is  positive).  Therefore,  we  have  that  p(ti)  >  p(t 2)  >  •  •  •  >  p(tn). 

Consider  two  servers — server  A  and  server  B — each  facing  the  state  (U, . . . ,  tn).  Suppose 
server  B  uses  a  policy  0,  in  which  0(U, . .  .,tn)  =  i  /  1.  Consider  a  policy  for  server  A 
as  follows:  Serve  customer  1  first.  If  server  A  finds  customer  1  to  be  white  and  no  red 
customer  has  left  (unserved)  yet,  then  (1)  if  customer  i  is  not  in  the  queue,  switch  to  policy 
0  thereafter;  (2)  if  customer  i  is  still  in  the  queue,  then  increment  the  state  variable  of 
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that  customer  by  ti  —  ti  (so  that  customer  will  be  treated  as  a  customer  who  has  spent  an 
additional  t\  —  ti  time  units  in  the  queue)  and  switch  to  policy  (j)  thereafter. 

Because  p(ti)  >  p(ti),  we  are  able  to  couple  customer  l’s  identity  and  customer  V s  identity 
in  queues  A  and  B  in  five  cases  as  follows  (see  Table  1  for  a  summary).  Define  a  random 
variable  /  to  indicate  which  case  takes  place,  and  let  X  denote  the  reward  for  server  A,  and 
Y  the  reward  for  server  B.  For  brevity,  we  use  A -i  to  denote  customer  i  in  queue  A,  and  so 
on. 

Table  1:  The  identities  of  customers  can  be  coupled  stochastically  in  five  cases,  used  in  the 
proof  of  Theorem  3.1. 


Probability 

Queue  A 

Queue  B 

Customer  1 

Customer  i 

Customer  1 

Customer  i 

(1  -P(*i))(l  ~p(U)) 

white 

white 

white 

white 

p(ti)p(U) 

red 

red 

red 

red 

(1  ~p(h))p(ti) 

red 

white 

white 

red 

(1  -p(ti))p(U) 

white 

red 

red 

white 

Pit i)  -  p(ti) 

red 

white 

red 

white 

1.  With  probability  (1  —  p(fi))(l  —  p(U)),  A-l,  A -i,  B-l,  and  B-i  are  all  white.  Because 
of  the  memoryless  property  of  exponential  distribution,  we  can  couple  A-l  and  B-z 
such  that  they  will  renege  at  the  same  time.  Similarly,  we  can  couple  A -i  and  B-l 
such  that  they  will  renege  at  the  same  time.  We  further  couple  the  service  times  for 
the  two  servers  such  that  the  kt\i  service  initiated  by  server  A  takes  the  same  amount 
of  time  as  the  kth  service  initiated  by  server  B,  k  =  1,2, . . ..  Finally,  we  couple  the 
identities  of  the  other  n— 2  customers  in  the  queue  and  their  respective  remaining  times 
to  renege,  as  well  as  the  arrival  times  of  future  customers,  their  identities,  and  their 
reneging  times.  By  doing  so,  we  can  see  that  both  queues  will  follow  the  same  sample 
path — except  that  customer  labels  1  and  i  are  swapped  in  the  two  queues  (server  A 
will  serve  customer  i  in  queue  A  if  and  only  if  server  B  serves  customer  1  in  queue 
B).  Consequently,  for  each  sample  path,  the  two  servers  will  earn  an  identical  reward; 
therefore,  E[X\I  =  1]  =  E[Y\I  =  1], 
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2.  With  probability  p(ti)p(ti),  A-l,  A-i,  B-l,  and  B -i  are  all  red.  By  coupling  the  sample 
paths  between  the  two  queues  exactly  the  same  way  as  in  case  1,  we  can  see  that 
servers  A  and  B  will  earn  an  identical  reward  in  each  sample  path.  Therefore,  we  can 
conclude  E[X\I  =  2]  =  E[Y \I  =  2], 

3.  With  probability  (1  —  p(ti))p(U),  A-l  and  B -i  are  red,  while  A-i  and  B-l  are  white. 
Similar  to  case  2,  we  can  conclude  E[X\I  =  3]  =  E[Y\I  =  3]. 

4.  With  probability  (1  —  A-l  and  B -i  are  white,  while  A-i  and  B-l  are  red. 

Similar  to  case  2,  we  can  conclude  E[X\I  =  4]  =  E[Y\I  =  4]. 

5.  With  probability  p(fi)—p(fj),  A-l  and  B-l  are  red,  while  A-i  and  B -i  are  white.  Because 
both  A-l  and  B-l  are  red,  we  can  couple  A-l  and  B-l  such  that  they  will  renege  at  the 
same  time.  Similarly,  we  can  couple  A-i  and  B-?'  such  that  they  will  renege  at  the  same 
time.  We  further  couple  the  service  times  for  the  two  servers  such  that  the  kth  service 
initiated  by  server  A  takes  the  same  amount  of  time  as  the  kth  service  initiated  by 
server  B,  k  —  1,  2, . . ..  Finally,  we  couple  the  identities  of  the  other  n  —  2  customers  in 
the  queue  and  their  respective  remaining  times  to  renege,  as  well  as  the  arrival  times 
of  future  customers,  their  identities,  and  their  reneging  times.  Consider  the  next  event 
that  occurs. 

(a)  If  the  next  event  to  occur  is  a  service  completion,  then  server  A  earns  1,  while 
server  B  may  eventually  earn  0,  r,  or  1.  The  probability  server  B  will  earn  0  or  r 
is  nonzero. 

(b)  If  the  next  event  to  occur  is  the  reneging  of  A-l  and  B-l,  then  server  A  earns  r 
while  server  B  earns  0. 

(c)  If  the  next  event  to  occur  is  the  reneging  of  A-i  and  B -i,  then  server  B  will  choose 
another  customer  to  serve.  At  that  point,  we  can  repeat  the  whole  stochastic 
coupling  argument  for  cases  1-5  listed  in  this  proof. 

(d)  If  the  next  event  to  occur  is  the  reneging  of  any  of  the  other  n  —  2  customers, 
then  both  servers  will  earn  0  if  that  customer  is  red.  If  that  reneging  customer  is 
white,  then  the  process  continues,  and  we  can  consider  the  next  event  and  repeat 
the  argument  in  cases  (a)-(e). 


(e)  If  the  first  event  to  occur  is  the  arrival  of  a  new  customer,  then  the  process 
continues,  and  we  can  consider  the  next  event  and  repeat  the  argument  in  cases 
a)-(e 

Consequently,  we  can  see  that  in  each  sample  path,  server  A  will  earn  a  reward  greater 
than  or  equal  to  what  server  B  will  earn.  Therefore,  E[X\I  =  5]  >  E[Y\I  =  5]. 


Taking  all  5  cases  together,  we  can  write  that 
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E[X\  -  E[Y]  =  ^2(E[x\I  —  k\  —  E[Y\I  =  k])  ■  P{I  =  k}  >  0. 

k= 1 

Hence,  any  policy  that  does  not  select  customer  1  cannot  be  optimal.  Because  there  are  only 
n  customers  to  choose  from,  it  must  be  optimal  to  select  customer  1. 

Finally,  because  the  preceding  argument  applies  each  time  the  server  becomes  available, 
it  follows  that  the  FCFS  rule  is  optimal.  □ 


Although  Theorem  3.1  holds  for  an  arbitrary  value  of  p,  we  are  particularly  interested  in 
the  case  when  p  — >  0,  because  terrorist  attacks  are  rare  events.  To  compute  the  expected 
reward  as  p  — >  0,  first  construct  a  queue  with  only  white  customers  arriving  according  to 
a  Poisson  process  with  rate  A,  and  then  let  a  red  customer  arrive  in  steady  state.  We  can 
obtain  a  closed-form  solution  for  the  expected  reward  if  the  service  time  distribution  is  also 
exponential. 

Suppose  Fs  is  exponential  with  rate  p.  With  white  customers  arriving  according  to  a 
Poisson  process  with  rate  A,  the  steady-state  probability  that  there  are  n  customers  in  the 
system  can  be  found  by  a  birth-death  process  (see  Chapter  2.9  in  Gross  and  Harris  [4]  for  a 
derivation),  and  is  given  by 


i  + Lentil 


for  n  =  0, 


(2) 


and 


n?=i( 


A 


IJ>+i0w ' 


for  n  =  1,2, 


The  queue  is  stable  as  long  as  0w  >  0,  regardless  of  the  values  of  p  and  A. 

If  a  red  customer  finds  n  white  customers  in  the  system  upon  arrival,  then  the  probability 
that  he  will  enter  service  before  reneging  is  the  probability  that  all  those  n  white  customers 
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depart — either  due  to  impatience  or  due  to  service  completion — before  the  red  customer 
reneges.  This  probability  can  be  obtained  by  the  memoryless  property  of  the  exponential 
distribution: 

-A-  /  /x  +  Mw  A 

i=i  v  h  +  iOw  +  Or  / 

Therefore,  with  the  FCFS  rule,  the  probability  that  the  red  customer  arriving  in  steady  state 
will  enter  service  before  reneging  is 


i+Er=,n*.,(^) 
i  +  E?=,n,./  J 


£ 

n=l 


H+iOw+Or ' 


i+Er=in?=i( 


A 


H+iQw  ' 


nr=i  i 


A 


A l+iO\y  ' 


1  +  £?=1  V/i  +  iOw  + Or  j  J 


n 


p  +  id 


w 


(3) 


Once  the  red  customer  enters  service,  we  can  deduce,  once  again  due  to  the  memoryless 
property  of  exponential  distribution,  that  the  service  will  complete  before  the  red  customer 
reneges  with  probability  // / (/i  +  Or).  Consequently,  the  expected  reward  for  the  FCFS  rule 
is  Equation  (3)  multiplied  by 

P  +  rOR 
p  +  Or 


3.2  Last-Come-First-Serve  (LCFS)  Rule 

With  the  LCFS  rule,  the  server  always  serves  the  customer  who  most  recently  joined  the 
queue.  Somewhat  surprisingly,  the  counterpart  of  Theorem  3.1  when  0R  >  0W  is  not  true 
even  if  Fr  is  also  exponential.  For  example,  if  there  is  only  one  customer  in  the  queue,  and 
that  customer  has  been  in  the  queue  for  a  long  time  (so  that  the  customer  is  most  likely 
white),  then  the  server  may  prefer  waiting  for  the  next  new  arrival  rather  than  serving  that 
very  old  customer,  as  shown  in  the  next  example. 

Example  3.1 

Suppose  A  =  /x  =  1,  Or  =  10,  0w  =  0.1,  and  p  =  0.8.  Consider  a  situation  when  there  is  only 
one  customer  in  the  queue — referred  to  as  customer  Z  throughout  this  example — who  joined 
the  queue  one  time  unit  ago.  According  to  Equation  (1),  customer  Z  is  a  red  customer  with 
probability  p(  1)  ~  0.0002. 

With  the  LCFS  rule,  the  server  initiates  service  with  customer  Z.  Let  X  denote  the 
reward  received  by  the  server  with  the  LCFS  rule.  Compute  P{ X  =  0}  by  conditioning  on 
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the  identity  of  customer  Z: 


P{X  =  0}  =  p(l)P{X  =  0|Z  is  red}  +  (1  - p{\))P{X  =  0|Z  is  white} 

.  .  . .  A  0  r 

>  0  +  (l-p(l))-  ,  -  ,  p 


(4) 


A  +  p  +  0\y  p  +  0\y  +  Or 

where  the  inequality  follows  because  conditional  on  customer  Z  being  white,  X  =  0  as  long 
as  the  following  three  events  occur  sequentially:  (1)  a  new  customer  arrives  before  customer 
Z  departs  (whether  due  to  impatience  or  due  to  service  completion);  (2)  the  new  customer 
is  red;  and  (3)  the  new  customer  reneges  (unserved)  before  customer  Z  departs. 

To  get  upper  bounds  for  P{X  =  r}  and  P{X  =  1},  respectively,  note  that  to  get 
a  positive  reward,  the  server  needs  to  select  a  red  customer  at  some  point.  Once  that 
happens,  there  is  still  a  chance  for  another  red  customer  to  renege  before  the  red  customer 
in  service  departs — either  due  to  impatience  (reward  r )  or  due  to  service  completion  (reward 
1).  However,  if  the  process  does  end  because  the  red  customer  in  service  departs,  then 
the  probabilities  whether  the  ending  is  due  to  impatience  or  due  to  service  completion  are 
proportional  to  their  respective  exponential  rates  0R  and  p.  Therefore,  we  can  conclude  that 

P{X  =  r}  _  9R 
P{X  =  1}  p  ' 

Hence,  we  have  that 

E[X]  =  1-P{X  =  l}  +  r-P{X  =  r}  +  0-P{X  =  0} 

+  v  nl  \  ^R 


=  (i  —  P{X  =  0}) 


< 


(l-(l-p(l)) 


h  + 

A 


A  +  /r  +  9\y 
0.05972(1  + lOr), 


+  r(l  —  P{X  =  0}) 

8r  A 

p.  +  9W  +  Or  )  n  +  0R 


V 


h  +  Or 
h  +  r0R 


(5) 


where  the  inequality  follows  from  Equation  (4). 

An  alternative  policy  is  for  the  server  to  stay  idle  until  a  new  customer  arrives,  and  then 
immediately  serve  the  newly  arrived  customer.  Let  Y  denote  the  reward  under  this  policy. 
Compute  P{Y  =  1}  by  conditioning  on  the  identity  of  customer  Z: 

P{Y  =  1}  =  p(l )P{Y  =  1 1 Z  is  red}  +  (1  —  p(l))P{Y'  —  1 1 Z  is  white} 

p 


>  0  +  (1  p(l))  p 


A  +  p  +  0 


R 
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where  the  inequality  follows  because  conditional  on  customer  Z  being  white,  the  server  will 
receive  a  reward  of  1  as  long  as  the  first  arrival  is  a  red  customer,  and  the  service  for  that 
red  customer  completes  before  the  red  customer  reneges  and  before  another  new  customer 
arrives.  Similarly,  we  have  that 


Therefore, 


P{Y  =  r}  >  (1  —  p(  1))  p 


Or 

A  +  p  +  Or 


E[Y]  =  1  •  P{Y  =  1}  +  r  ■  P{Y  =  r}  +  0  ■  P{Y  =  0} 

>  (1~p(1))prry^ 

«  0.06665(1  +  lOr). 


(6) 


From  Equations  (5)  and  (6),  we  can  conclude  that  E[Y]  >  E[ X]  for  any  r  e  [0, 1],  which 
implies  that  the  LCFS  rule  is  not  optimal.  □ 


Similar  to  the  FCFS  rule  discussed  in  Section  3.1,  we  let  p  — >  0  and  calculate  the  expected 
reward  under  the  LCFS  rule.  As  p  — >  0,  we  can  End  this  probability  by  first  constructing  a 
queue  with  only  white  customers  and  letting  a  red  customer  arrive  in  steady  state.  We  first 
calculate  the  probability  that  the  red  customer  arriving  in  steady  state  will  enter  service 
before  reneging. 

Note  that  if  the  server  is  idle  when  a  red  customer  arrives,  then  the  red  customer  enters 
service  immediately.  If  the  server  is  busy  when  a  red  customer  arrives,  then  with  the  LCFS 
rule,  the  current  number  of  white  customers  in  the  system  is  irrelevant  to  whether  the  red 
customer  will  enter  service  before  reneging.  In  this  case,  we  construct  a  Markov  chain  to 
represent  the  state  of  the  system  when  a  red  customer  is  present.  Denote  by  k  the  state 
if  the  server  is  busy  with  a  white  customer,  and  there  are  k  —  1  white  customers  in  the 
queue  behind  the  red  customer,  k  —  1,  2, . . ..  Let  the  state  become  0  when  the  red  customer 
enters  service,  and  —1  when  the  red  customer  reneges  before  entering  service.  Note  that  by 
definition,  states  0  and  —1  are  absorbing,  and  that  the  Markov  chain  starts  in  state  1. 

Let  a*,,  k  —  1, . . . ,  oo,  denote  the  probability  that  the  Markov  chain  in  state  k  will  ever 
enter  state  k  —  1  before  entering  state  —1  (the  red  customer  reneges  unserved).  We  need  to 
determine  oq,  the  probability  that  a  red  customer  will  enter  service  before  reneging  if  the 
server  is  busy  upon  his  arrival. 
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To  obtain  ay,  we  first  find  an  expression  for  aq  by  conditioning  on  whether  the  next 
event  is  a  new  arrival,  a  departure  of  a  white  customer,  or  the  departure  of  the  unserved  red 
customer: 


A 


Oik  = 


9  r  +  A  +  /i  +  k9w 


■  ack+iOik  + 


H  +  kO 


w 


9  Ft  +  A  +  /i  +  k9\y 


•  1  + 


9 


R 


9  Ft  +  A  +  /i  +  k9\y 


•o, 


for  k  —  1,  2, . . ..  Solving  for  oq.  yields 


/i  +  k9\y 

_ 

9  Ft  +  A  +  p  +  —  Aafc+i 

Because  G  [0, 1],  the  preceding  implies  that 


(7) 


/i  +  k9\y  fi  +  k9\y  .  . 

+  A  +  /i  +  +  /i  + 

Consequently,  we  can  choose  a  large  value  of  k,  use  Equation  (8)  to  bound  a*,,  and  then  use 
Equation  (7)  to  recursively  compute  the  bounds  for  ccfc_ i,  2, . . . ,  ai.  Because  the  bounds 
converge  very  quickly,  we  can  approximate  ol\  satisfactorily. 

Finally,  we  can  compute  the  probability  that  the  red  customer  enters  service  before 
leaving  under  the  LCFS  rule  by 


1  •  P{server  idle  in  steady  state}  +  aq  ■  P (server  busy  in  steady  state} 

1/1 
+  cr  i  ( 1  — 


i  +  Er=in?=i( 


A 


1-L+iOw  • 


i+£r=in?=i( 


A 


H+iOw  • 


(9) 


where  the  steady-state  probability  is  given  by  Equation  (2).  With  the  same  reason  given  at 
the  end  of  Section  3.1,  the  expected  reward  with  the  LCFS  rule  is  Equation  (9)  multiplied 
by 

h  +  tOr 
11  +  9  Ft 


3.3  Random  Selection  (RS)  Rule 

Another  service  rule  of  interest  is  the  RS  rule,  in  which  the  server,  when  becoming  available, 
randomly  selects  a  customer  in  the  queue  to  serve.  If  both  reneging  times  are  exponentially 
distributed  and  9pt  =  9W,  then  all  three  rules — FCFS,  LCFS,  and  RS — perform  equally  well 
for  two  reasons:  (1)  each  customer  in  the  queue  has  a  probability  p  of  being  red  regardless 
of  the  amount  of  time  he  has  spent  in  the  queue;  and  (2)  the  remaining  times  to  renege 
for  all  customers  in  the  queue  are  independent  and  identically  distributed  because  of  the 
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memoryless  property  of  the  exponential  distribution.  If  9R  ^  9W)  we  would  expect  that  the 
performance  of  the  RS  rule  lies  between  those  of  the  FCFS  and  the  LCFS  rules. 

As  p  — >  0,  we  can  formulate  a  Markov  chain  to  compute  the  expected  reward  of  the  RS 
rule  similar  to  that  in  Sections  3.1  and  3.2.  We  omit  the  derivation.  To  compare  the  three 
service  rules,  note  that  in  order  for  the  server  to  earn  a  positive  reward,  the  red  customer 
arriving  in  steady  state  (of  a  system  that  consists  of  only  white  customers)  needs  to  enter 
service  before  reneging.  If  the  red  customer  does  enter  service,  then  the  expected  reward 
becomes  (/x  +  r9R)/(p  +  dR)  as  derived  at  the  end  of  Section  3.1.  Therefore,  the  relative 
performance  among  the  three  rules  is  independent  of  r — the  reward  of  partial  success.  For 
this  reason,  to  compare  the  three  service  rules  when  p  — >  0,  we  plot  in  Figures  1  and  2  the 
probability  that  the  arriving  red  customer  will  ever  enter  service  before  reneging — namely 
Equation  (3)  for  the  FCFS  rule  and  Equation  (9)  for  the  LCFS  rule. 

As  seen  in  Figures  1  and  2,  the  FCFS  rule  is  the  best  of  the  three  when  9R  <  9w  (in 
this  case,  the  FCFS  is  optimal  by  Theorem  3.1),  while  the  LCFS  rule  is  the  best  when 
9R  >  0\v-  Examples  of  arenas  for  the  case  0R  <  9w  include  the  vending  machine  corners, 
walkways,  stairs,  and  parking  lots,  where  a  typical  visitor  tends  to  leave  fairly  soon.  In  this 
case,  although  the  FCFS  rule  is  optimal,  its  performance  is  quite  sensitive  to  the  changes  in 
either  9W  or  9R.  Examples  for  the  case  0R  >  9W  include  department  stores,  public  parks, 
picnic  areas,  and  other  places  where  a  typical  visitor  tends  to  spend  a  long  time.  In  this  case, 
although  the  LCFS  rule  is  not  optimal,  its  performance  is  relatively  robust  to  the  change  in 
9w  and  9R,  especially  when  9R  «  9w ■  This  observation  suggests  that  if  the  value  of  9R  is 
highly  uncertain — as  9\y  is  typically  much  easier  to  estimate — then  the  LCFS  rule  may  be 
preferred. 

3.4  Preemptive  Service 

Our  queueing  model  assumes  that  the  service  is  nonpreemptive  because  the  screening  process 
of  a  suspect  cannot  be  interrupted.  Although  preemptive  service — whereby  the  server  can 
interrupt  the  current  screening  and  switch  to  another  customer  upon  a  new  arrival  or  any 
departure — may  not  be  a  practical  assumption,  we  are  interested  in  such  a  variation  from  a 
theoretical  standpoint.  This  subsection  presents  a  theorem  that  complements  Theorem  3.1 
in  the  nonpreemptive  service  case. 
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Figure  1:  Comparison  of  three  service  rules  when  the  expected  reneging  time  for  a  white 
customer  varies.  Fr,  Fw,  and  Fs  are  all  exponential;  p  — >  0,  A  =  2,  1  /Or  =  6,  and  1/p,  =  2. 

Theorem  3.2  If  the  service  is  preemptive,  and  both  Fr  and  Fw  are  exponential  with  re¬ 
spective  rates  Or  <  9W  (respectively,  0R  >  0W),  then  the  FCFS  (respectively,  LCFS)  rule  is 
optimal  for  an  arbitrary  arrival  process  if  Fs  is  exponential. 

Proof:  We  prove  the  optimality  of  the  LCFS  rule  when  Or  >  $w,  while  the  proof  for  the 
optimality  of  the  FCFS  rule  when  Or  <  0w  follows  a  similar  argument. 

Consider  two  servers — server  A  and  server  B — each  facing  the  state  (ti,t2, .  ■  ■  ,tn),  such 
that  t\  >  t2  >  ■  ■  ■  >  tn.  Suppose  server  B  uses  a  policy  0,  in  which  . . .  ,tn)  =  i  ^  n. 
Consider  a  feasible  policy  for  server  A  as  follows:  Start  service  with  customer  n  and  continue 
until  an  arrival  or  a  departure;  thereafter  switch  to  policy  (f  unless  that  event  is  the  departure 
of  a  red  customer  (in  which  case  the  process  ends).  Let  X  denote  the  reward  for  server  A, 
and  Y  the  reward  for  server  B.  To  prove  the  theorem,  we  will  show  that  E[ X]  >  E’fY]. 
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Figure  2:  Comparison  of  three  service  rules  when  the  expected  reneging  time  for  a  red 
customer  varies.  FR,  Fw,  and  Fs  are  all  exponential;  p  — >  0,  A  =  2,  l/0w  =  6,  and  1/ p  =  2. 


Recall  from  Equation  (1)  that  p(t)  denotes  the  probability  that  a  customer  is  red  if  he 
has  spent  t  time  units  in  the  queue.  Because  dR  >  9W  and  ti  >  t2  >  ■  ■  ■  >  tn,  it  follows 
that  p(ti)  <  p(t 2)  <  •  •  •  <  p(tn).  We  couple  customer  n’s  identity  and  customer  i’s  identity 
in  queues  A  and  B  in  the  following  four  cases,  and  define  a  random  variable  /  to  indicate 
which  case  takes  place.  For  brevity,  we  use  A -i  to  denote  customer  i  in  queue  A,  and  so  on. 

1.  With  probability  p(tn)p(ti),  A -i,  A -n,  B-i,  and  B-n  are  all  red:  Because  the  times  to 
renege  for  red  customers  follow  independent  exponential  distribution  with  rate  0R,  it 
follows  that  E[X\I  =  1]  =  E[Y\I  =  1], 

2.  With  probability  (1  —p(tn))(  1  —p(ti)),  A -i,  A -n,  B-i,  and  B-n  are  all  white:  Similar  to 
case  1,  because  the  times  to  renege  for  white  customers  follow  independent  exponential 
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distribution  with  rate  6\y ,  it  follows  that  E[X\I  =  2]  —  E[Y\I  =  2], 

3.  With  probability  (1  —  p(tn))p(ti),  A -n  and  B-n  are  white,  while  A -i  and  B -i  are  red: 
We  couple  A-n  and  B-n  such  that  they  will  renege  at  the  same  time.  Similarly,  we 
couple  A-i  and  B -i  such  that  they  will  renege  at  the  same  time.  We  further  couple 
the  service  times  for  the  two  servers  such  that  the  hth  service  initiated  by  server  A 
takes  the  same  amount  of  time  as  the  kill  service  initiated  by  server  B,  k  =  1,2,.... 
Consider  the  next  event  that  takes  place.  If  the  next  event  is  service  completion,  then 
X  =  0  and  Y  —  1.  If  the  next  event  is  the  reneging  of  A-i  and  B -i,  then  X  =  0  and 
Y  —  r.  If  the  next  event  is  neither  of  the  above,  then  the  two  servers  will  receive  the 
same  reward  eventually.  Therefore, 

E[Y\I  =  3]  -  E[X\I  =  3]  >  0. 

4.  With  probability  (1  —  p(ti))p(tn),  A-n  and  B-n  are  red,  while  A-i  and  B -i  are  white: 
We  can  couple  the  two  systems  similarly  to  case  3.  Because  the  situation  is  reverse  to 
that  in  case  3,  however,  it  follows  that 

E[X\I  =  4]  -  E[Y\I  =  4]  =  E[Y\I  =  3]  -  E[X\I  =  3]  >  0. 


With  these  4  cases,  we  can  conclude  that 

4 

E[X]-E[Y]  =  J2(ElX\I  =  k\-E[Y\I  =  k])-P{I  =  k} 

k=  1 


=  (E[X\I  =  4]  -  E[Y\I  =  4])  •  ((1  -  piUMQ  -  (1  -  p(tnMti)) 


=  ( E[X\I  =  4]  -  E[Y\I  =  4])  •  (p(tn)  -  pit t))  >  0, 


and  the  result  follows. 


□ 


Note  that  contrary  to  Theorem  3.1,  Theorem  3.2  does  not  hold  for  an  arbitrary  service 
time  distribution  Fs.  A  counterexample  can  be  constructed  intuitively  as  follows.  Suppose 
that  Fr  and  Fw  are  exponential  with  respective  rates  6r  <  0w,  and  that  Fs  has  a  decreasing 
failure  rate — that  is,  fs(t)/Fs(t)  decreases  in  t,  for  t  >  0 — so  that  the  longer  a  customer  has 
been  in  service,  the  longer  (stochastically)  his  remaining  service  time  becomes  (for  example, 
see  Proposition  9.1.3  in  Ross  [18]).  Granted,  with  the  FCFS  rule,  the  server  always  serves 
the  customer  who  has  the  highest  probability  of  being  red.  However,  after  serving  the  same 
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customer  for  a  long  time  without  a  conclusion,  the  remaining  service  time  tends  to  be  even 
longer  (in  the  regular  stochastic  sense).  At  that  point,  the  server  may  prefer  to  switch  to 
another  customer  for  a  fresh  service  time,  even  though  this  other  customer  is  less  likely  a 
red  customer. 


4  General  Reneging  Time  Distributions 

This  section  presents  the  case  when  Fr  and  Fw  do  not  follow  exponential  distributions.  In 
Subsection  4.1,  we  explain  why  it  is  difficult  to  ford  the  optimal  policy  for  general  reneging 
time  distribution.  In  Subsection  4.2,  we  develop  a  heuristic  policy.  In  Subsection  4.3,  we  use 
Monte  Carlo  simulation  to  numerically  evaluate  the  heuristic  policy. 


4.1  Special  Cases  and  Counterexamples 


We  first  investigate  whether  we  can  relax  the  exponential  assumption  on  Fr  and  Fw  so  that 
the  FCFS  rule  remains  optimal  under  weaker  conditions.  Intuitively,  for  the  FCFS  rule  to 
be  optimal,  two  conditions  need  to  hold:  (a)  the  longer  a  customer  has  spent  in  the  queue, 
the  more  likely  he  is  a  red  customer;  and  (b)  the  longer  a  white  customer  has  spent  in  the 
queue,  the  sooner  he  tends  to  leave  the  queue  according  to  his  reneging  distribution  so  as  to 
save  the  server’s  time. 

For  condition  (a)  to  hold,  we  need  pit)  in  Equation  (1)  to  increase  in  t.  Calculus  shows 
that  for  pit)  to  increase  in  t,  a  sufficient  condition  is  that  Fr  has  a  smaller  failure  rate  than 
Fw ;  in  other  words, 


/r(C  <  fw{t) 


for  t  >  0, 


(10) 


FR(t)  J  Fw(ty 

where  /  and  g  are  the  density  functions  and  Fr  and  Fw  are  the  tail  distribution  functions. 
For  condition  (b)  to  hold,  we  need  the  random  variable  iW  —  t\W  >  t)  to  decrease  in  t  in 
the  regular  stochastic  sense,  where  W  denotes  a  random  variable  with  distribution  function 
Fw-  That  is,  we  need 


P{W  -t  >  s\W  >t}  = 


Fw{t  +  s) 
Fw(t) 


to  decrease  in  t  for  all  s  >  0.  Calculus  shows  that  an  equivalent  condition  for  the  preceding 
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to  hold  is  for  F\v  to  have  an  increasing  failure  rate;  that  is, 


„ — ,  ,  increases  in  t,  for  t>0.  (11) 

F\v{t) 

Before  imposing  a  condition  on  the  failure  rate  of  Fr,  consider  two  special  cases,  r  =  0  and 
r  =  1,  to  further  simplify  the  problem. 

The  case  r  =  0 

In  the  special  case  r  =  0,  the  server  gets  no  reward  for  a  partial  success,  so  it  is  useless  to 
initiate  service  for  a  red  customer  if  the  server  cannot  complete  the  service  before  the  red 
customer  reneges.  In  order  to  make  the  FCFS  rule  more  attractive,  it  seems  intuitive  to 
impose  the  assumption  that  the  longer  a  red  customer  has  spent  in  the  queue,  the  longer 
his  additional  reneging  time  tends  to  be.  In  other  words,  we  need  the  random  variable 
(R  —  1 1  R  >  t)  to  increase  in  t  in  the  regular  stochastic  sense,  where  R  denotes  a  random 
variable  with  distribution  function  Fr.  Similar  to  the  derivation  of  Equation  (11),  the 
preceding  is  equivalent  to  the  following: 

decreases  in  t,  for  t  >  0.  (12) 

FR{t) 

A  plausible  conjecture  is  that  when  r  =  0,  the  FCFS  rule  is  optimal  if  Equations  (10)-(12) 
hold.  The  following  example,  however,  disapproves  this  conjecture. 

Example  4.1 

Let  r  —  0.  Suppose  that  the  reneging  time  of  a  white  customer  is  exponentially  distributed 
with  rate  equal  to  1,  and  that  of  a  red  customer  has  the  following  failure  rate  function: 

fn(t)  f  !,  for  0  <  t  <  2, 

Fn(t)  \  0.01,  for  t  >  2. 

It  is  straightforward  to  verify  that  Equations  (10)— (12)  are  satisfied.  Also  suppose  that 
p  =  0.5,  and  the  service  time  distribution  F$  is  deterministic  and  equal  to  0.5.  In  addition, 
let  A  =  0.0001  so  that  the  effect  of  future  arrivals  is  negligible.  Consider  a  scenario  when 
the  server  finds  two  customers  in  the  queue  with  tx  —  2  and  t2  —  1. 

Compare  two  service  orders  1,  2  (FCFS)  and  2,  1.  If  both  customers  are  white,  then  with 
either  service  order  the  process  continues  after  both  customers  depart.  If  at  least  one  of  the 
two  customers  is  red,  then  the  process  ends  when  (or  before )  both  customers  depart.  Using 
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Monte  Carlo  simulation,  it  turns  out  that,  conditional  on  at  least  one  customer  is  red,  the 
expected  reward  is  0.671  for  service  order  1,  2,  and  0.734  for  service  order  2,  1  (standard 
error  less  than  3  x  10~5).  Because  it  is  optimal  to  serve  customer  2,  the  FCFS  rule  is  not 
optimal.  □ 


To  intuitively  understand  Example  4.1,  note  that  p[ti)  =  p(t2 )  =  0.5.  If  both  customers 
are  red,  then  it  does  not  make  any  difference  to  start  with  either  customer,  because  to  get  a 
reward  of  1  (r  =  0),  the  server  must  complete  the  service  before  either  red  customer  reneges. 
If  customer  1  is  red  and  customer  2  is  white,  then  customer  l’s  additional  reneging  time 
follows  an  exponential  distribution  with  mean  equal  to  100  time  units.  By  starting  with 
customer  2,  the  server  most  likely  will  be  able  to  serve  out  customer  1  after  customer  2 
reneges.  On  the  other  hand,  if  customer  1  is  white  and  customer  2  is  red,  the  server  has  a 
much  smaller  chance  to  serve  out  customer  2  by  starting  with  customer  1. 


The  case  r  —  1 


Consider  the  second  special  case  when  r  —  1,  so  that  a  partial  success  is  as  good  as  a  full 
success.  In  this  case,  the  server  always  hopes  the  customer  in  service  to  renege  as  soon 
as  possible.  To  make  the  FCFS  rule  attractive  when  r  =  1,  it  is  intuitive  to  impose  the 
assumption  that  the  longer  a  red  customer  has  spent  in  the  queue,  the  sooner  he  tends  to 
renege.  In  other  words,  we  assume  that 


PR(t ) 


increases  in  t, 


for  t  >  0, 


(13) 


which  is  the  opposite  to  Equation  (12).  Hence,  a  plausible  conjecture  is  that  when  r  =  1, 
the  FCFS  rule  is  optimal  if  Equations  (10),  (11),  and  (13)  hold.  The  next  example,  however, 
disapproves  this  conjecture. 

Example  4.2 

Let  r  =  1.  Suppose  that  Fr  and  F\y  are  identical  with  the  following  failure  rate  function: 


Ht) 


fw(t) 

Fw(t) 


0, 

1, 

10000, 


for  0  <  t  <  2, 
for  2  <  t  <  4, 
for  t  >  4. 
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It  is  straightforward  to  verify  that  Equations  (10),  (11),  and  (13)  are  satisfied.  Also  suppose 
that  the  probability  of  a  red  customer  is  p  =  0.5.  Because  Fr  and  Fw  are  identical,  the 
server  cannot  learn  about  a  customer’s  identity  from  the  amount  of  time  the  customer  has 
spent  in  the  queue.  Hence,  p{t)  =  p  =  0.5  for  all  t  >  0. 

Suppose  there  are  three  customers  with  U  =  2.99,  f2  =  2.01,  and  t3  =  1,  and  that  the 
service  time  distribution  F$  is  deterministic  and  equal  to  1.  In  addition,  assume  A  =  0.0001 
so  that  the  effect  of  future  arrivals  is  negligible.  Compare  two  service  orders  1,  2,  3  (FCFS) 
and  2,  1,  3.  If  all  three  customers  are  white,  then  with  either  service  order  the  process 
continues  after  all  three  customers  depart.  If  at  least  one  of  the  three  customers  is  red,  then 
the  process  ends  when  (or  before )  all  three  customers  depart.  Using  Monte  Carlo  simulation, 
it  turns  out  that,  conditional  on  at  least  one  customer  is  red,  the  expected  reward  is  0.720 
for  the  service  order  1,  2,  3,  and  0.752  for  the  service  order  2,  1,  3  (standard  error  less  than 
5  x  10”5).  Therefore,  it  is  better  to  start  with  customer  2  rather  than  with  customer  1,  and 
the  FCFS  rule  is  not  optimal.  □ 

To  gain  some  intuition  about  this  example,  first  note  that  the  failure  rate  function  remains 
a  constant  for  2  <  t  <  4,  and  the  service  time  is  deterministic  and  equal  to  1.  Because 
[ti,ti  +  1]  C  [2,4)  for  i  =  1,2,  the  time  it  takes  for  the  server  to  become  available  is 
identically  distributed  regardless  of  whether  the  server  starts  with  customer  1  or  customer 
2.  In  addition,  if  the  server  starts  with  customer  1,  the  probability  that  customer  2  is  still 
in  the  queue  when  the  server  becomes  available  is  the  same  as  the  probability  that  customer 
1  is  still  in  the  queue  if  the  server  starts  with  customer  2.  Consequently,  the  number  of 
customers  between  customers  1  and  2  that  the  server  can  serve  by  following  the  order  2,  1, 
3  is  identically  distributed  to  that  number  when  the  server  follows  the  order  1,  2,  3. 

However,  by  starting  with  customer  2,  the  time  it  takes  for  the  server  to  become  available 
for  customer  3  is  stochastically  smaller  than  by  starting  with  customer  1,  because  as  soon  as 
a  customer  spends  4  time  units  in  the  queue,  he  will  leave  almost  immediately.  Consequently, 
by  starting  with  customer  2,  the  server  has  a  better  chance  to  serve  customer  3. 

Examples  4.1  and  4.2  show  that  even  in  the  special  cases  when  r  =  0  and  r  =  1,  the 
FCFS  rule  is  not  optimal  under  some  plausible  conditions.  To  determine  the  optimal  policy 
for  an  arbitrary  r  can  only  be  more  difficult.  Therefore,  we  next  turn  our  attention  to  a 
heuristic  policy. 
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4.2  Heuristic  Policy 


To  develop  a  heuristic  policy,  consider  a  related  problem  in  which  the  process  does  not  stop 
when  a  red  customer  departs  for  the  first  time.  Instead,  the  process  continues  indefinitely,  and 
the  server’s  goal  is  to  maximize  the  long-run  average  reward  earned  from  each  red  customer. 
Note  that  as  p  — >  0,  this  related  problem  and  the  original  problem  become  equivalent, 
because  in  either  problem  the  objective  function  is  to  maximize  the  expected  reward  earned 
from  a  red  customer  that  arrives  in  steady  state  of  a  queueing  system  consisting  of  only 
white  customers  that  arrive  according  to  a  Poisson  process  with  rate  A. 

Based  on  this  observation,  we  propose  a  heuristic  policy  where  the  server  chooses  the 
customer  with  the  highest  reward  rate — the  ratio  between  the  expected  reward  and  the 
expected  time  spent  if  the  customer  is  served.  Let  R ,  W,  and  S  denote  random  variables 
with  respective  probability  distribution  functions  FR,  Fw,  and  F$.  Suppose  a  customer  has 
spent  t  time  units  in  the  queue,  then  serving  that  customer  yields  a  reward  rate  equal  to 

_  E  [reward  received  from  serving  a  customer  who  has  spent  t  time  units  in  queue] 
if  [time  spent  on  serving  a  customer  who  has  spent  t  time  units  in  queue] 
pit)  ■  (. P{R  -t>  S\R>t}  +  rP{R  —  t<  S\R  >  t})  +  (1  -  p(t))  •  0 
p(t)E[min(R  -  t,  S)\R  >  t]  +  (1  -  p(f))£[min(IE  -  t,  S)\W  >  t]  ’ 

where  p(t)  is  given  by  Equation  (1),  the  probability  a  customer  is  red  if  he  has  spent  t  time 
units  in  the  queue.  As  p  — »  0,  we  can  compare  the  reward  rate  between  any  two  customers 


by 


lim  lA 

p—*o  7 (t2) 


Fnjt^jr  +  (1  -  r)P{R  -  F  >  S\R  >  t,}) 
Fwit^EiminiW  -  tuS)\W  >  R 


' FR(t2)(r  +  (1  -  r)P{R  -  t2  >  S\R  >  t2}) 
Fw(t2)E[mm(W  -  t2,  S)\W  >  t2] 

Therefore,  we  define  a  score  function  for  a  t-time-unit-old  customer  as 

,  ,  _  FR(t)(r  +  (1  -  r)P{R  -t>S\R>  t}) 

S[  }  ~  Fw(t)E[mm(W  -  t,  S)\W  >  t] 

and  let  the  server  choose  the  customer  who  has  the  highest  score. 

To  further  compute  Equation  (15),  we  calculate 

r  oo 

P{R-t  >  SIR  >  t}  =  /  P{R  -  t  >  x\R  >  t}fs(x)dx 

Jo 


(14) 


(15) 


r°°  FR{t  +  x ) 
3  FR{t) 


fs(x)dx, 


(16) 
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and 


r  oo 

E[mm(W -t,S)\W  >  t]  =  /  P{mm(W  -  t,  S)  >  x\W  >  t}dx 

Jo 

r  oo 

=  /  P{W  —  t  >  x  and  S  >  x\W  >  t)dx 

Jo 

r  oo 

=  /  P{W-t  >  x\W  >t}P{S  >  x\W  >t}dx 

Jo 

f°°  Fw(t  +  X ) 


Fw(J) 


-Fs{x)dx. 


(17) 


(18) 


Consequently,  putting  Equations  (15)-(17)  together  gives 

=  rFfi(t)  +  (1  -  r)  /0°°  +  x)fs(x)dx 

f™  Fw(t  + x)Fs(x)dx 

The  score  in  Equation  (18)  is  computed  for  each  customer  individually,  based  on  the 
time  a  customer  has  spent  in  the  queue.  One  advantage  of  this  score  is  that  it  is  easy  to 
compute.  In  practice,  we  can  compute  the  score  s(t)  for  all  values  of  t  beforehand,  which 
allows  easy  implementation  in  real  time.  Observe  that  a  customer’s  score  does  not  depend 
on  the  number  of  other  customers  in  the  queue  nor  the  customer  arrival  rate  A.  Therefore, 
this  heuristic  cannot  be  optimal  in  general.  In  particular,  when  the  traffic  is  relatively  light, 
the  server  should  take  into  account  the  possibility  of  becoming  idle  when  the  queue  is  empty, 
rather  than  always  choosing  the  customer  that  yields  the  highest  reward  rate.  Hence,  in 
a  light-traffic  system  it  is  possible  to  devise  a  policy  that  is  specifically  tailored  for  given 
distributions  Fr,  Fw ,  and  F$.  On  the  other  hand,  in  a  surveillance  system  under  heavy 
traffic — the  case  we  expect  to  see  in  applications — there  are  many  customers  to  choose  from 
each  time  the  server  becomes  available.  Because  the  server  would  be  kept  busy  most  of 
the  time,  it  should  focus  on  selecting  the  customer  that  yields  the  highest  reward  rate. 
Consequently,  we  expect  our  heuristic  policy  to  be  effective  in  a  heavy-traffic  system. 

When  Fr,  Fw,  and  S  follow  exponential  distributions  with  respective  rates  9r,  Ow,  and 
/i,  the  score  function  in  Equation  (18)  becomes 


(t)  =  (0W  +  n)  (^F±^)e-d>n-0W)t 


' R  +  h 

When  Or  <  Ow,  the  preceding  increases  in  t,  so  the  heuristic  policy  coincides  with  the 
FCFS  rule — the  optimal  policy  according  to  Theorem  3.1.  When  Or  >  Ow,  the  heuristic 
policy  coincides  with  the  LCFS  rule — an  intuitively  good  policy  although  not  optimal  due 
to  Example  3.1. 
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4.3  Numerical  Experiments 

This  subsection  presents  a  numerical  example.  Let  Fr  follow  the  Erlang  distribution  with 
shape  parameter  6  and  scale  parameter  1,  and  F\v  the  Erlang  distribution  with  shape  pa¬ 
rameter  2  and  scale  parameter  3.  We  choose  the  two  distributions  to  have  the  same  expected 
value,  namely  6,  because  if  they  are  much  different,  the  heuristic  policy  resembles  either  the 
FCFS  rule  or  the  LCFS  rule.  We  also  choose  the  distributions  so  that  Fw  has  a  larger 
variance,  because  the  purposes  of  visit  are  much  diversified  for  white  customers.  The  choice 
of  Erlang  distribution  is  primarily  due  to  its  unimodal  density  function.  The  service  time 
is  largely  deterministic,  but  we  let  F$  follow  a  uniform  distribution  to  allow  for  a  small 
variance. 

Instead  of  varying  r,  we  plot  the  score  function  in  Equation  (18)  for  r  —  0  in  Figure  3, 
and  for  r  —  1  in  Figure  4.  Because  s(t)  is  a  linear  function  in  r,  for  an  arbitrary  r  e  [0, 1], 
the  score  function  is  basically  a  weighted  average  between  these  two  extreme  cases. 

In  the  case  r  =  0,  the  server  does  not  earn  any  reward  if  a  red  customer  reneges  during 
service.  As  seen  in  Figure  (3),  when  F$  ~  U (0.5, 1.5),  the  score  function  s(t)  largely  coincides 
with 

pit)  _  _  FR(t)  _  _  FR(t) 

p(  o)  pFR(t)  +  (i  -p)Fw(t)  Fw(ty  asp 

In  other  words,  when  the  service  time  is  small,  the  server  selects  the  next  customer  primarily 
based  on  the  likelihood  of  the  customer  being  red,  because  most  likely  the  service  will 
complete  before  the  customer  reneges.  When  the  service  time  is  large,  however,  it  becomes 
less  desirable  to  serve  a  customer  who  has  spent  longer  in  the  queue  (Erlang  distribution 
has  an  increasing  failure  rate),  because  the  chance  of  reneging  during  service  becomes  larger. 
When  Fs  ~  17(7.5,8.5),  the  service  time  is  so  great  that  the  heuristic  policy  coincides  with 
the  LCFS  rule. 

In  the  case  r  =  1,  the  server  earns  a  reward  of  1  for  a  partial  success.  As  seen  in  Figure  4, 
s(t)  still  follows  a  similar  shape  to  FR(t)/ F\v(t)  when  Fs  ~  77(0.5,1.5).  When  the  service 
time  becomes  larger,  the  peak  of  s(t)  shifts  to  the  right  slightly.  A  customer  who  has  spent 
a  longer  time  in  the  queue  becomes  more  attractive  because  the  server  can  save  time  if  it  is 
a  white  customer;  in  addition,  the  server  does  no  need  to  complete  service  to  earn  a  reward 
of  1  if  it  is  a  red  customer. 

We  next  compare  the  performance  of  the  heuristic  policy  with  the  other  three  naive  service 
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Time  a  customer  has  spent  in  the  queue  t 


Figure  3:  Score  function  s(t )  for  r  =  0;  FR  ~  Erlang(6, 1),  Fw  ~  Erlang(2,3),  and  Fs 
follows  four  different  uniform  distributions. 


rules — FCFS,  LCFS,  and  RS  rules — using  the  same  example.  To  simulate  the  performance 
of  a  policy,  note  that  as  p  — >  0,  a  red  customer  will  arrive  in  steady  state  of  a  queueing 
system  that  consists  of  only  white  customers  arriving  according  to  a  Poisson  process  with 
rate  A.  However,  it  is  inefficient  to  collect  only  one  estimate  each  time  we  generate  a  steady 
state.  To  overcome  this  issue,  we  generate  a  sample  path  of  the  queueing  system  where  white 
customers  arrive  according  to  a  Poisson  process  with  rate  A  for  the  first  n  (a  large  number) 
arrivals,  and  let  the  server  process  customers  according  to  a  given  service  rule — FCFS,  LCFS, 
RS,  or  heuristic.  After  generating  the  sample  path,  we  turn  our  attention  to  each  customer 
one  at  a  time.  For  the  jth  arriving  customer,  define  the  random  variable  Zj  as  the  reward 
the  server  would  have  earned  had  customer  j  been  a  red  customer,  while  all  other  customers 
remain  white.  Hence,  our  estimator  is  (Xp=i  Zj)/n. 
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Figure  4:  Score  function  s(t)  for  r  =  1;  FR  ~  Erlang(6, 1),  Fw  ~  Erlang(2,  3),  and  Fs  follows 
four  different  uniform  distributions. 


Our  simulation  algorithm  uses  steady-state  simulation  to  collect  multiple  estimates  in 
a  single  simulation  run.  There  are  two  issues  related  to  a  steady-state  simulation.  First, 
there  is  initial  bias  because  the  system  is  not  in  steady  state  when  we  start  the  simulation 
with  an  empty  queue.  Second,  the  random  variables  Zj  and  Zj+ 1  are  not  independent.  If 
Zj  =  0,  it  becomes  more  likely  for  the  queue  to  have  many  customers,  which  in  turn  makes 
Zj+ 1  more  likely  to  also  take  on  value  0.  To  resolve  these  two  issues,  we  allow  a  prolonged 
warm-up  period  before  collecting  data,  and  use  batch  means  to  estimate  the  standard  error 
of  our  estimate;  see,  for  example,  Law  and  Kclton  [12],  We  choose  the  batch  size  so  that 
with  probability  close  to  1  the  first  customers  in  consecutive  batches  will  never  coexist  in 
the  system. 

In  the  simulation  experiment,  we  choose  F$  ~  17(1.5,2.5),  and  simulate  three  cases  for 
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r  =  0,  0.5,  and  1.  Table  2  compares  the  expected  reward  as  p  — >  0  for  four  policies  when 
the  arrival  rate  A  varies  from  1  to  5.  In  each  case,  we  use  a  sufficiently  large  n  so  that 
the  standard  error  is  about  10“3  of  the  estimate.  We  choose  the  performance  of  the  RS 
rule  as  the  benchmark,  and  report  the  performance  of  the  other  three  rules  as  ratios  to  the 
benchmark. 

Table  2:  Expected  reward  for  different  policies  as  p  0;  FR  ~  Erlang(6, 1),  Fw  ~ 
Erlang(2,3),  and  Fs  ~  U(  1.5,  2.5). 


Ratio  to  RS  Rule 


r 

A 

RS 

RS 

FCFS 

LCFS 

Heuristic 

1 

0.464 

1.000 

0.927 

1.076 

1.096 

2 

0.217 

1.000 

0.617 

1.210 

1.233 

0 

3 

0.141 

1.000 

0.417 

1.243 

1.273 

4 

0.104 

1.000 

0.299 

1.260 

1.294 

5 

0.083 

1.000 

0.228 

1.270 

1.304 

1 

0.538 

1.000 

1.028 

0.987 

1.082 

2 

0.264 

1.000 

0.821 

1.028 

1.170 

0.5 

3 

0.173 

1.000 

0.610 

1.033 

1.197 

4 

0.128 

1.000 

0.473 

1.039 

1.211 

5 

0.102 

1.000 

0.374 

1.042 

1.223 

1 

0.612 

1.000 

1.106 

0.922 

1.132 

2 

0.311 

1.000 

0.963 

0.900 

1.230 

1 

3 

0.205 

1.000 

0.747 

0.890 

1.257 

4 

0.153 

1.000 

0.586 

0.888 

1.270 

5 

0.122 

1.000 

0.473 

0.886 

1.278 

As  seen  in  Table  2,  the  heuristic  policy  always  yields  the  highest  expected  reward.  In 
addition,  the  heuristic  policy’s  relative  improvement  over  the  RS  rule  gradually  increases  as 
A  increases.  This  observation  is  not  surprising,  as  we  argued  in  Section  4.2  that  the  heuristic 
policy  is  particularly  suitable  in  a  heavy-traffic  system,  in  which  the  server  can  often  select 
a  high-score  customer  from  a  full  spectrum  of  customers. 
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The  FCFS  rule  performs  well  when  A  =  1,  especially  in  the  case  r  =  1.  As  seen  in 
Figure  4,  the  score  function  s(t)  ( Fs  ~  U(  1.5,  2.5))  increases  when  t  is  small.  When  A  is 
small,  often  there  are  only  a  few  customers  who  are  new  to  the  system,  so  the  FCFS  rule 
often  selects  the  same  customer  as  does  the  heuristic  policy.  When  A  =  5,  however,  often 
the  queue  is  full  of  customers,  many  of  which  have  spent  a  long  time  in  the  queue.  In  that 
case,  the  FCFS  rule  would  select  a  customer  that  has  been  in  the  queue  for  a  long  time, 
whereas  the  heuristic  policy  tends  to  select  a  customer  who  has  spent  about  4  time  units  in 
the  queue.  For  r  =  0  and  r  =  0.5,  the  FCFS  rule  also  performs  poorly  for  A  =  5,  because 
s(t)  is  decreasing  for  larger  values  of  t. 

The  LCFS  rule  performs  relatively  well  in  the  case  r  =  0,  because  in  Figure  3  the  score 
function  s(t)  (Fs  ~  17(1.5,  2.5))  is  largely  decreasing  in  t,  so  the  LCFS  rule  and  the  heuristic 
policy  often  makes  the  same  decision.  In  the  case  r  =  1,  s(t)  in  Figure  4  is  unimodal  with 
the  maximum  occurring  about  4.  When  A  increases,  the  LCFS  rule  often  selects  a  customer 
that  just  entered  the  queue,  while  the  heuristic  policy  tends  to  select  a  customer  that  has 
spent  about  4  time  units  in  the  queue.  Therefore,  the  performance  of  the  LCFS  rule  drops 
as  A  increases. 


5  Concluding  Remarks 

In  this  paper  we  developed  a  single-server  queueing  model  with  impatient  customers  to  study 
a  surveillance  system  aimed  at  detecting  terrorists  in  real  time.  Two  types  of  customers — 
terrorist  and  nonterrorist — arrive  at  the  system,  but  a  customer  does  not  reveal  his  identity 
upon  arrival.  The  server,  however,  can  draw  inference  about  a  customer’s  likely  identity 
based  on  the  time  the  customer  stays  in  the  system.  We  presented  a  few  cases  in  which  the 
optimal  policy  can  be  explicitly  determined,  and  studied  a  heuristic  policy  that  performs 
well  in  a  heavy-traffic  system. 

Because  our  study  focused  on  the  scheduling  aspect  of  the  screening  operation,  we  as¬ 
sumed  that  the  surveillance  system  has  perfect  sensitivity  and  perfect  specificity.  If  the 
surveillance  system  would  erroneously  classify  a  terrorist  as  a  nonterrorist  (false  negative) 
with  a  certain  probability,  then  the  performance  of  the  surveillance  system  described  in  this 
paper  would  simply  be  discounted  by  that  probability.  If  false  positive  errors  are  also  possi¬ 
ble,  then  the  actions  taken  by  the  authorities  would  incur  a  social  cost  associated  with  the 


disruption  of  normal  daily  life.  This  cost,  however,  is  typically  much  smaller  than  that  of  a 
successful  terrorist  attack. 

There  are  a  few  related  research  directions  that  can  follow  from  our  study.  First,  it  is 
possible  to  extend  the  queueing  model  to  allow  multiple  servers  and  more  than  two  types 
of  customers  (catching  a  terrorist  is  more  rewarding  than  catching  a  criminal  fugitive). 
Second,  the  probability  of  classification  errors  can  be  modeled  as  a  function  of  the  time  a 
target  is  under  surveillance.  The  longer  the  surveillance  system  monitors  a  target,  the  more 
likely  the  classification  will  be  correct.  In  this  case,  the  service  time  becomes  a  controlled 
variable  rather  than  a  random  parameter.  We  believe  that  mathematical  modeling  along 
these  research  lines  has  the  potential  to  advance  the  effort  on  counterterrorism  and  homeland 
security. 
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